Aircraft Detection¶
- Get Pleiades imagery for the given airports
- Execute tiling and aircraft detection blocks via parallel jobs
Visualize the results
The example costs around 1700 UP42 credits
Setup¶
Import required libraries
In [2]:
import up42
import geopandas as gpd
import rasterio
from rasterio.plot import show
import matplotlib.pyplot as plt
from shapely.geometry import box
Configure areas of interest
In [3]:
aoi_txl = {"type":"FeatureCollection","features":[{"type":"Feature","properties":{},
"geometry":{"type":"Polygon","coordinates":[[[13.286740779876709,52.5509016976356],
[13.300495147705078,52.5509016976356],
[13.300495147705078,52.556890079685594],
[13.286740779876709,52.556890079685594],
[13.286740779876709,52.5509016976356]]]}}]}
aoi_muc = {"type":"FeatureCollection","features":[{"type":"Feature","properties":{},
"geometry":{"type":"Polygon","coordinates":[[[11.789016723632812,48.348577346994944],
[11.809401512145996,48.348577346994944],
[11.809401512145996,48.360155725059116],
[11.789016723632812,48.360155725059116],
[11.789016723632812,48.348577346994944]]]}}]}
aois = [{'title': 'TXL', 'geometry': aoi_txl},
{'title': 'MUC', 'geometry': aoi_muc}]
Authenticate with UP42
In [4]:
#up42.authenticate(project_id="123", project_api_key="456")
up42.authenticate(cfg_file="config.json")
up42.settings(log=False)
2020-11-11 22:59:10,921 - Got credentials from config file. 2020-11-11 22:59:11,197 - Authentication with UP42 successful! 2020-11-11 22:59:11,198 - Logging disabled - use up42.settings(log=True) to reactivate.
Catalog Search¶
Search cloudfree Pleiades image for the two aois and visualise the quicklooks.
In [5]:
catalog = up42.initialize_catalog()
for aoi in aois:
print("\n---------" + aoi["title"] + "---------\n")
search_paramaters = catalog.construct_parameters(geometry=aoi['geometry'],
start_date="2020-04-01",
end_date="2020-04-30",
sensors=["pleiades"],
max_cloudcover=10,
sortby="acquisitionDate",
ascending=False,
limit=3)
search_results = catalog.search(search_paramaters)
# Download & Visualise quicklooks
catalog.download_quicklooks(image_ids=search_results.id.to_list(), sensor="pleiades")
display(search_results.head())
catalog.plot_quicklooks(figsize=(18,5), titles=search_results.scene_id.to_list())
# Select least cloud scene for further workflow
aoi["scene_id"] = search_results.scene_id.to_list()[0]
---------TXL---------
100%|██████████| 3/3 [00:01<00:00, 1.73it/s]
| geometry | id | acquisitionDate | constellation | providerName | blockNames | cloudCoverage | up42:usageType | providerProperties | scene_id | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | POLYGON ((13.20640 52.58098, 13.20616 52.45458... | c5497393-8f8d-4367-9527-c39344e220fe | 2020-04-28T10:31:35Z | PHR | oneatlas | [oneatlas-pleiades-fullscene, oneatlas-pleiade... | 2.31 | [DATA, ANALYTICS] | {'commercialReference': 'SO20025725', 'acquisi... | DS_PHR1B_202004281031350_FR1_PX_E013N52_0513_0... |
| 1 | POLYGON ((13.21800 52.58082, 13.21860 52.45684... | 07e5eaba-7830-4aee-8c27-6b69fa1da89f | 2020-04-23T10:19:52Z | PHR | oneatlas | [oneatlas-pleiades-fullscene, oneatlas-pleiade... | 0.00 | [DATA, ANALYTICS] | {'commercialReference': 'SO20022129', 'acquisi... | DS_PHR1B_202004231019525_FR1_PX_E013N52_0513_0... |
| 2 | POLYGON ((13.20760 52.58274, 13.20828 52.45502... | 11616f18-4002-44fc-bfeb-571c6657ccf4 | 2020-04-17T10:16:45Z | PHR | oneatlas | [oneatlas-pleiades-fullscene, oneatlas-pleiade... | 0.40 | [DATA, ANALYTICS] | {'commercialReference': 'SO20017653', 'acquisi... | DS_PHR1A_202004171016453_FR1_PX_E013N52_0513_0... |
---------MUC---------
100%|██████████| 3/3 [00:01<00:00, 2.09it/s]
| geometry | id | acquisitionDate | constellation | providerName | blockNames | cloudCoverage | up42:usageType | providerProperties | scene_id | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | POLYGON ((11.62946 48.39067, 11.62967 48.29748... | 0e324d95-47df-48f7-9a49-76a9aaa93362 | 2020-04-16T10:25:42Z | PHR | oneatlas | [oneatlas-pleiades-fullscene, oneatlas-pleiade... | 0.00 | [DATA, ANALYTICS] | {'commercialReference': 'SO20017376', 'acquisi... | DS_PHR1B_202004161025425_FR1_PX_E011N48_1009_0... |
| 1 | POLYGON ((11.63674 48.40241, 11.63713 48.31271... | b9d55cf6-6a6b-4a41-9998-0fc69e1d4515 | 2020-04-11T10:14:13Z | PHR | oneatlas | [oneatlas-pleiades-fullscene, oneatlas-pleiade... | 0.08 | [DATA, ANALYTICS] | {'commercialReference': 'SO20016407', 'acquisi... | DS_PHR1B_202004111014133_FR1_PX_E011N48_1009_0... |
| 2 | POLYGON ((11.64703 48.39837, 11.64731 48.30483... | 6573a62a-819b-4d2e-b27d-8bbd904e6549 | 2020-04-10T10:20:58Z | PHR | oneatlas | [oneatlas-pleiades-fullscene, oneatlas-pleiade... | 0.05 | [DATA, ANALYTICS] | {'commercialReference': 'SO20015930', 'acquisi... | DS_PHR1A_202004101020584_FR1_PX_E011N48_1009_0... |
In [10]:
# Optional: Select ideal scenes manually
aois[0]["scene_id"] = "DS_PHR1B_202004281031350_FR1_PX_E013N52_0513_01239"
aois[1]["scene_id"] = "DS_PHR1B_202004161025425_FR1_PX_E011N48_1009_00822"
Download selected Pleiades images for aois¶
In [14]:
up42.settings(log=True)
project = up42.initialize_project()
# Increase the parallel job limit for the project.
#project.update_project_settings(max_concurrent_jobs=10)
2020-11-11 18:13:41,729 - Logging enabled (default) - use up42.settings(log=False) to disable. 2020-11-11 18:13:42,369 - Initialized Project(name: Aircraft Detection TXL-MUC, project_id: 0a448563-a2e6-41f3-abf6-737d036309ee, description: , createdAt: 2020-11-11T16:45:02.388745Z)
Create or update a workflow for the aircraft detection
In [15]:
workflow = project.create_workflow("Aircraft detection", use_existing=True)
2020-11-11 18:13:57,529 - Getting existing workflows in project ... 2020-11-11 18:13:57,836 - Got 0 workflows for project 0a448563-a2e6-41f3-abf6-737d036309ee. 0it [00:00, ?it/s] 2020-11-11 18:13:58,168 - Created new workflow: ea37cc26-396b-4d67-b22e-855786bd23af
Add or update workflows tasks
In [23]:
#up42.get_blocks(basic=True)
input_tasks= ['oneatlas-pleiades-aoiclipped', 'tiling', 'orbital_pleiades_aircraft']
workflow.add_workflow_tasks(input_tasks=input_tasks)
workflow
2020-11-11 18:29:12,006 - Added tasks to workflow: [{'name': 'oneatlas-pleiades-aoiclipped:1', 'parentName': None, 'blockId': '18d09f1a-3197-4c27-a15a-54d099c31435'}, {'name': 'tiling:1', 'parentName': 'oneatlas-pleiades-aoiclipped:1', 'blockId': '3e146dd6-2b67-4d6e-a422-bb3d973e32ff'}, {'name': 'orbital_pleiades_aircraft:1', 'parentName': 'tiling:1', 'blockId': '4f2f3438-d31b-4872-ab15-ce50160dd70e'}]
Out[23]:
Workflow(name: Aircraft detection, workflow_id: ea37cc26-396b-4d67-b22e-855786bd23af, description: , createdAt: 2020-11-11T17:13:58.145399Z, project_name: Aircraft detection, workflow_tasks: {'oneatlas-pleiades-aoiclipped:1': '2.1.1', 'tiling:1': '2.2.3', 'orbital_pleiades_aircraft:1': '1.1.2-public'}
Run jobs in parallel¶
Construct workflow input parameters & run jobs
In [24]:
input_parameters_list = []
for aoi in aois:
input_parameters = workflow.construct_parameters(geometry=aoi['geometry'],
geometry_operation="bbox",
scene_ids=[aoi["scene_id"]])
input_parameters['tiling:1']['tile_width'] = 1024
input_parameters['tiling:1']['tile_height'] = 1024
input_parameters_list.append(input_parameters)
input_parameters_list
Out[24]:
[{'oneatlas-pleiades-aoiclipped:1': {'limit': 1,
'zoom_level': 18,
'max_cloud_cover': 100,
'panchromatic_band': False,
'ids': ['DS_PHR1B_202004281031350_FR1_PX_E013N52_0513_01239'],
'bbox': [13.286740779876709,
52.5509016976356,
13.300495147705078,
52.556890079685594]},
'tiling:1': {'tile_width': 1024,
'tile_height': 1024,
'match_extents': False,
'output_prefix': '',
'augmentation_factor': 1,
'discard_empty_tiles': True},
'orbital_pleiades_aircraft:1': {}},
{'oneatlas-pleiades-aoiclipped:1': {'limit': 1,
'zoom_level': 18,
'max_cloud_cover': 100,
'panchromatic_band': False,
'ids': ['DS_PHR1B_202004161025425_FR1_PX_E011N48_1009_00822'],
'bbox': [11.789016723632812,
48.348577346994944,
11.809401512145996,
48.360155725059116]},
'tiling:1': {'tile_width': 1024,
'tile_height': 1024,
'match_extents': False,
'output_prefix': '',
'augmentation_factor': 1,
'discard_empty_tiles': True},
'orbital_pleiades_aircraft:1': {}}]
In [25]:
jobs = workflow.run_jobs_parallel(input_parameters_list=input_parameters_list)
2020-11-11 18:30:25,609 - Selected input_parameters: {'oneatlas-pleiades-aoiclipped:1': {'limit': 1, 'zoom_level': 18, 'max_cloud_cover': 100, 'panchromatic_band': False, 'ids': ['DS_PHR1B_202004281031350_FR1_PX_E013N52_0513_01239'], 'bbox': [13.286740779876709, 52.5509016976356, 13.300495147705078, 52.556890079685594]}, 'tiling:1': {'tile_width': 1024, 'tile_height': 1024, 'match_extents': False, 'output_prefix': '', 'augmentation_factor': 1, 'discard_empty_tiles': True}, 'orbital_pleiades_aircraft:1': {}}.
2020-11-11 18:30:29,248 - Created and running new job: 1e7f5b2d-9a9c-4155-85ea-58b36a1e5d59
2020-11-11 18:30:29,575 - Selected input_parameters: {'oneatlas-pleiades-aoiclipped:1': {'limit': 1, 'zoom_level': 18, 'max_cloud_cover': 100, 'panchromatic_band': False, 'ids': ['DS_PHR1B_202004161025425_FR1_PX_E011N48_1009_00822'], 'bbox': [11.789016723632812, 48.348577346994944, 11.809401512145996, 48.360155725059116]}, 'tiling:1': {'tile_width': 1024, 'tile_height': 1024, 'match_extents': False, 'output_prefix': '', 'augmentation_factor': 1, 'discard_empty_tiles': True}, 'orbital_pleiades_aircraft:1': {}}.
2020-11-11 18:30:32,473 - Created and running new job: 8f7801de-d00a-4f59-8e4e-a61750148b29
2020-11-11 18:30:32,810 - Tracking job status continuously, reporting every 20 seconds...
2020-11-11 18:30:56,512 - Job is RUNNING! - 1e7f5b2d-9a9c-4155-85ea-58b36a1e5d59
2020-11-11 18:31:20,901 - Job is RUNNING! - 1e7f5b2d-9a9c-4155-85ea-58b36a1e5d59
2020-11-11 18:31:44,278 - Job is RUNNING! - 1e7f5b2d-9a9c-4155-85ea-58b36a1e5d59
2020-11-11 18:32:05,658 - Job is RUNNING! - 1e7f5b2d-9a9c-4155-85ea-58b36a1e5d59
2020-11-11 18:32:28,017 - Job is RUNNING! - 1e7f5b2d-9a9c-4155-85ea-58b36a1e5d59
2020-11-11 18:32:51,364 - Job is RUNNING! - 1e7f5b2d-9a9c-4155-85ea-58b36a1e5d59
2020-11-11 18:33:14,892 - Job finished successfully! - 1e7f5b2d-9a9c-4155-85ea-58b36a1e5d59
2020-11-11 18:33:19,893 - Tracking job status continuously, reporting every 20 seconds...
2020-11-11 18:33:41,591 - Job is RUNNING! - 8f7801de-d00a-4f59-8e4e-a61750148b29
2020-11-11 18:33:52,284 - Job finished successfully! - 8f7801de-d00a-4f59-8e4e-a61750148b29
Download & Visualise results¶
In [43]:
data_results_paths, detection_results = [], []
for job in jobs:
data_task, _, detection_task = job.get_jobtasks()
data_paths = data_task.download_results()
data_results_paths.append([p for p in data_paths if p.endswith(".tif")])
detection_results.append(detection_task.get_results_json())
2020-11-11 18:40:54,657 - Getting job tasks: 1e7f5b2d-9a9c-4155-85ea-58b36a1e5d59 2020-11-11 18:40:56,159 - Downloading results of jobtask 2e5bc39f-6033-4c5b-9087-2231d8586ca4 2020-11-11 18:40:56,161 - Download directory: /Users/christoph.rieke/repos/up42-py/examples/project_0a448563-a2e6-41f3-abf6-737d036309ee/job_1e7f5b2d-9a9c-4155-85ea-58b36a1e5d59/jobtask_2e5bc39f-6033-4c5b-9087-2231d8586ca4 16346it [00:00, 419474.03it/s] 2020-11-11 18:40:58,315 - Download successful of 3 files to output_directory '/Users/christoph.rieke/repos/up42-py/examples/project_0a448563-a2e6-41f3-abf6-737d036309ee/job_1e7f5b2d-9a9c-4155-85ea-58b36a1e5d59/jobtask_2e5bc39f-6033-4c5b-9087-2231d8586ca4': ['usage.json', 'data.json', '46ebeb50-7109-4d0e-bc4c-377df12b5d04.tif'] 2020-11-11 18:40:58,745 - Retrieved 17 features. 2020-11-11 18:40:58,746 - Getting job tasks: 8f7801de-d00a-4f59-8e4e-a61750148b29 2020-11-11 18:41:00,169 - Downloading results of jobtask 5cd8d78c-ba26-4f3f-8fbe-57a02ac1f12f 2020-11-11 18:41:00,170 - Download directory: /Users/christoph.rieke/repos/up42-py/examples/project_0a448563-a2e6-41f3-abf6-737d036309ee/job_8f7801de-d00a-4f59-8e4e-a61750148b29/jobtask_5cd8d78c-ba26-4f3f-8fbe-57a02ac1f12f 41688it [00:00, 467632.89it/s] 2020-11-11 18:41:04,617 - Download successful of 3 files to output_directory '/Users/christoph.rieke/repos/up42-py/examples/project_0a448563-a2e6-41f3-abf6-737d036309ee/job_8f7801de-d00a-4f59-8e4e-a61750148b29/jobtask_5cd8d78c-ba26-4f3f-8fbe-57a02ac1f12f': ['6e8adf74-021f-4ef8-b97a-3dbdce2e5c2b.tif', 'usage.json', 'data.json'] 2020-11-11 18:41:05,038 - Retrieved 76 features.
In [48]:
for i, (paths, detection) in enumerate(zip(data_results_paths, detection_results)):
with rasterio.open(paths[0]) as src:
fig, ax = plt.subplots(figsize=(18, 18))
planes = gpd.GeoDataFrame.from_features(detection, crs={'init': "EPSG:4326"})
planes = planes.to_crs(epsg=3857)
planes.geometry = planes.geometry.buffer(0.0001)
planes.geometry = planes.geometry.apply(lambda geo:box(*geo.bounds))
show(src.read(), transform=src.transform, ax=ax, title=f"{aois[i]['title']}: {planes.shape[0]} planes detected")
planes.plot(ax=ax, facecolor=(0,0,0,0), edgecolor='red', linewidth=2)
plt.show()
In [ ]: